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Abstract 


A set of global, monthly rainfall products has been intercompared to understand the quality 
and utility of the estimates. The products include 25 observational (satellite-based), four 
model and two climatological products. The results of the intercomparison indicate a very 
large range (factor of two or three) of values when all products are considered. The range 
of values is reduced considerably when the set of observational products is limited to those 
considered quasi-standard. The model products do significantly poorer in the tropics, but 
are competitive with satellite-based fields in mid-latitudes over land. Over ocean, products 
are compared to frequency of precipitation from ship observations. The evaluation of the 
observational products point to merged data products (including rain gauge information) as 
providing the overall best results. 
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1. Introduction 


Over the past six years several intercomparisons of satellite precipitation algorithms, such 
as the first and second Precipitation Intercomparison Projects (PIP-1 and PIP-2), and AIP 
(Algorithm Intercomparison Project) -1,-2 and -3 have aided the development and use of 
global satellite precipitation products. A summary of results from the AEP program is given 
by Ebert et al. (1996). The PIP-1 project, which is closest in form to the current PIP-3 
being discussed, is discussed in Barrett et al. (1994). The PIP-2 intercomparison which 
focused on instantaneous estimates based on passive microwave observations is 
summarized by Smith et al. (1998). The current PIP-3 follows the success of these 
previous efforts, but puts increased emphasis on evaluation of quasi-standard, satellite- 
based, global, monthly precipitation fields. 

The PIP-3 project was sponsored by NASA through the WetNet Project and was endorsed 
by the Global Precipitation Climatology Project (GPCP) of the WCRP/GEWEX Program. 
This article summarizes the results of the PIP-3 Workshop, held at College Park, MD, and 
the pre-workshop and post-workshop analysis carried out with the submitted data sets. 
Sixty scientists attended the workshop representing numerous organizations involved in 
precipitation analysis from both an observational and modeling perspective. Additional 
information on the project and the workshop and detailed results can be found at the PIP-3 
homepage address of http://ghrc.msfc.nasa.gov/pip3 . A compact disc (CD) of the results, 
data sets, imag'es, etc. is also available. 
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2. PIP-3 Objective and Approach 


The objective of PIP-3 is to determine the utility of the current quasi-standard global, 
monthly precipitation products to the climate modeling and diagnostic community and the 
potential improvement expected with the latest satellite algorithms. The user community 
needs recommendations on accuracy and usefulness for a variety of applications including 
global model validation and climate monitoring and diagnostics. The algorithm community 
needs information on the future requirements of the user community. PIP-3 was designed 
to produce an evaluation of the current products and facilitate the exchange of information 
on future directions. 

The monthly, global rain totals and rainfall frequencies submitted by the participants were 
evaluated against surface validation data sets, including an atoll gauge data set, ocean 
precipitation frequency, and land gauge data sets. 

A full year (1992) was analyzed to test annual cycle retrieval. January and July of 1991, 
1992, and 1993 were included to allow evaluation of interannual variations. August 1987 
from the PIP-1 period was also evaluated to seek evidence of algorithm improvement 
during the past five years. Products using Special Sensor Microwave / Imager (SSM/I), 
geosynchronous infrared, Microwave Sounding Unit (MSU), TIROS Operational Vertical 
Sounder (TOVS) data, merged analysis schemes and composite microwave algorithms 
were included as well as prototype Tropical Rainfall Measuring Mission (TRMM) and 
Earth Observing System (EOS) microwave algorithms. Precipitation fields calculated from 
General Circulation Models (GCMs) were also included in the comparison. 

The evaluation statistics were kept fairly simple and consist mainly of bias, root mean 
square error, and correlation versus the validation data. The satellite-based products were 
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examined with regard to their overall reasonableness (e.g., rainfall maxima in the right 
place and of reasonable intensity), freedom from artifacts (e.g., unnatural coastline 
precipitation features) and the statistical comparison to the validation data. 


3. Description of Products and Validation Data Sets 

Table 1 summarizes the thirty-one products. The twenty-five observational products, four 
model-based products and the two climatologies were intercompared with each other and 
the validation data sets in terms of monthly rainfall statistics during 1992, interannual 
variations among the January’s and July’s of 1991, 1992 and 1993 and the frequency of 
precipitation over the ocean (monthly and annual statistics). 

The twenty-five observational products were divided into two groups for certain aspects of 
the analysis. The Quasi -Standard (Q-S) products were identified as those already in use by 
the modeling/diagnostic community, available for long, multi-year periods and readily 
available from archives, etc. These products tended to be the more mature products among 
the observational submissions. The remaining observational products were categorized as 
Experimental (EXP). All but one of the Experimental products were based on SSM/I data 
alone and typically were produced especially for the PIP-3 activity. Seven SSM/I-based 
products and one other product were ocean only estimates. Of the nine Quasi-Standard (Q- 
S) products, three were SSM/I-based, one each based on Microwave Sounding Unit 
(MSU) data, TIROS Operational Vertical Sounder (TOVS) data, and geosynchronous IR 
data and three were merged estimates using a combination of satellite observations or a 
combination of satellite and surface gauge observations. 
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The four model products included calculated fields from the reanalysis efforts at the 
European Centre for Medium-range Weather Forecasting (ECMWF), the National Centers 
for Environmental Prediction (NCEP) and the NASA Goddard Space Flight Center 
(GSFC) and a climatological average of the Atmospheric Model Intercomparison Project 
(AMIP) climate models (Lau et al.,1996). 

The descriptions of the submitted products can be found on the noted web site or project 
CD, or through the references in Table 1. 

The validation data set used in the PIP-3 study was accumulated from a number of sources. 
Land areas: 

i) the Global Precipitation Climatology Centre (GPCC) gauge product (Rudolf, 1994) was 
taken as the base validation data set. The raw data product (as opposed to the 
climatological-corrected product) was used. However, there were some noteable areas that 
have little data. Outside Western Europe the number of gauges is sparse, even in countries 
such as the US and Australia where gauge coverage is known to be good. 

ii) the Surface Reference Data Centre (SRDQ (Huffman et al., 1997) data set was seen as 
the most accurate of the data sets. Each 2.5 degree box used numerous gauges to generate 
the rainfall estimate. Unfortunately, it was also the least comprehensive, being only 
available for a few 2.5 degree boxes. 

iii) supplementary gauge data was sought for USA, Australia and South Africa to fill in 
some of the voids in the GPCC data set. The gauge data from these areas was interpolated 
and mapped to the 2.5 degree resolution. In addition, to boost the number of gauges in the 


6 


tropical region, data from the Amazon region was incorporated from the Amazon River 
Basin Precipitation data set at the Oak Ridge National Laboratory. The land gauge data sets 
were merged on the basis of the SRDC product having top priority, followed by the 
supplementary gauge data where the number of gauges exceeded that of the GPCC data, 
and lastly, the GPCC data set. 

A subset of this data base was chosen for the validation of the algorithm products in order 
to achieve a representative geographic data set. The selection was based upon the number 
of gauges available per 2.5 degree box, by the number of boxes within each climatic 

region, and the proximity to other boxes: Figure 1 shows the distribution of the validation 
boxes chosen. 

The interannual validation data set was based upon selected areas of four contiguous 2.5 x 
2.5 degree boxes in order to reduce the noise from both the validation and algorithm data. 
Areas were chosen as representative samples of the different climatic regimes. These areas 
can be seen in Fig. 1 as the three groups of four outlined boxes in the U.S. and Australia. 


Ocean: 

Oceanic validation data, especially gauge data, is very limited. For the PIP-3 study, atoll 
raingauge data from the Comprehensive Pacific Rainfall Data Base (CPRDB; Morrissey et 
ah, 1995) were used. Data from the atolls were collected, quality controlled and mapped by 
the Environmental Verification and Analysis Center (EVAC) at the University of 
Oklahoma. The data were grouped into three regions reflecting the seasonal characteristics 

of the rainfall data, namely north of 5° N, 10° S - 5° N and south of 10° S. Figure 1 
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shows the locations of the atoll validation grid boxes. For the interannual comparisons the 
sum of the 10° S - 5° N boxes was used (as shown by the outline in Fig. 1). 


Frequency of Precipitation: 

Data from the Comprehensive Ocean-Atmosphere Data Set (COADS; see Petty 1995) was 
used as validation data over the oceans, and was prepared by one of the authors (Petty). 
The COADS data set, comprised of ship observations of present and past weather, were 
used to determine the occurrence, or frequency, of precipitation. Due to the sparse nature 
of the observations in certain parts of the globe, data from the period 1958 to 1991 were 
used to generate an average, and therefore should be treated as a climatological average of 
the frequency of precipitation. The fractional-time-precipitating was derived from ship 
reports falling within a latitude-longitude window centered on the grid box in question. 
The dimensions of the window were chosen so as to achieve an adequate statistical sample 
without unnecessarily smoothing real gradients in rainfall distributions. Two sets of 
validation data were generated, one using all the COADS data with observations reporting 
all precipitation, except drizzle, and another set reporting all precipitation except drizzle and 
snow. The latter data set was included on the basis that estimates of precipitation from the 
passive microwave sensor would not include drizzle and snowfall. 

4. Intercomparison of Monthly Rainfall Totals 

The global, monthly rainfall total maps for 1992 were examined and intercompared in a 
number of ways and against the validation data sets over the Western Pacific Ocean atolls 
and over land. They were also examined for artifacts and for reasonableness over areas 
where no validation data sets exist, for example in the mid-latitude oceans. Four examples 
(of the different product types) of monthly maps for July 1992 are shown in Fig. 2. All 
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four examples display the main features of a July precipitation map. The Inter-Tropical 
Convergence Zone (ITCZ) stretches across the Pacific and Atlantic Oceans and northern 
South America. The Asian summer monsoon is producing rainfall maxima over India, 
Indochina and adjoining water areas. Northern Australia is in its dry season. In the 
tropics, the four example maps show very similar patterns and similar magnitudes. In mid- 
latitudes oceanic maxima are evident, with varying intensities. For example, the three 
maxima (the top, right panel (model)) at approximately 40°S east of Africa, in the central 
Pacific and east of South America are evident in the experimental and quasi-standard 
examples, but with different magnitudes. The noisiness of the Experimental product is due 
to the limited sampling with the low-orbit satellite. A similar noisy pattern is evident in the 
quasi-standard example at ocean latitudes above 40°. 

4.1 Zonal annual totals over water and land 

Zonal averages of the annual (1992) total over the ocean of each of the 25 observational 
products indicate a wide variation among the products, both in the tropics and in middle 
and high latitudes. The mean value and range of values at each latitude are plotted in Fig. 
3a. All the products generally capture the tropical maximum, the sub-tropical minima and 
the mid-latitude maxima. However, among all the observational products the peak value in 

the Inter-Tropical Convergence Zone (ITCZ) at 8°N varies from 1300 to 3200 mm for the 

annual total. This large variability among the estimates is also evident in mid-latitudes with 
values ranging from 900 to 1800 mm in the Northern Hemisphere maximum, with 
additional outliers above and below those values. In very high latitudes, for example at 

60°S, the range becomes an order of magnitude, going from 100 to 1000 mm. 
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At first this large variability among the estimates is disconcerting. However, if the set of 
products is limited to the Quasi-Standard data products, the range of values decreases 
significantly. This effect can be seen in Fig. 3b, which shows the standard deviation 
among the estimates as a function of latitude for both the 25 observational estimates and the 
subset of eight Q-S products. This decrease in the variability as we go from all to the 
Quasi-standard products mainly reflects the maturity of the products. In addition, there is 
some interdependence among the Q-S products because of merged products using some of 
the same input fields. Many of the products in the Experimental group were based on early 
versions of retrieval algorithms and due to errors, or perhaps a lack of tuning, some of 
these products produced values outside the range of reasonableness. These facts point to 

the need for the user community to exercise caution in selecting products with which to 
work. 

Fig. 4a compares the average of all the observational products with the two climatologies. 
The tropical peak in the Legates/Wilmott climatology is significantly larger than the 
observational products or the Jaeger climatology. In fact, the zonal totals also indicate that 
the Legates/Wilmott climatology has higher values in the ITCZ as compared to all the Q-S 
products. This difference is mainly related to the large peak found in the climatology in the 
east-central Pacific Ocean in the ITCZ during the Northern Hemisphere summer. None of 
the observational products support the existence of this feature, although they are looking at 
only one year of data. In the dry, subtropic zone in the Southern Hemisphere oceans the 
Legates/Wilmott (L/W) climatology also carries significantly higher values than all the 
observational products. The difference here is due mainly to the lesser westward extent 
•from the South American and African coasts of the subtropic minima in the climatology as 

compared to the satellite estimates. In mid-latitudes (poleward of 40°) the mean of all the 

observational products is significantly less than the climatologies. The observational mean 
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is dominated by SSM/I-based products which seem to have a tendency for underestimation 
at these latitudes, as is discussed in a later section. 

Still higher than the LAV climatology in the ITCZ is the ECMWF reanalysis model result 
with a peak annual total of 3100 mm (Fig. 4b). Between 0° and 10°S the ECMWF model 
result is significantly larger than the Q-S products, being 2500 mm and nearly double the 
mean of the Q-S products at 5°S. This difference in zonal total is related to very strong 

precipitation features in the ECMWF calculations in the central Pacific Ocean and over the 
Indian Ocean. The satellite estimates do not support the magnitude of the maxima, 
especially in the Indian Ocean. In mid-latitudes the models generally agree with the shape 
and magnitude of the climatologies and the mean of the Q-S products. 

The zonally-averaged totals over land show similar results with the tropical peak at the 
Equator ranging from 1300 to 2800 mm, but with a smaller range when only “mature” 
products were considered. Products with errors related to misidentifying desert surface as 
rain and products with other artifacts were identified with the zonal annual averages. 

4.2 Validation of estimates with Western Pacific Ocean atoll data set 

The Western Pacific Ocean atoll rainfall data set (Morrissey, et al. 1995) was used to 
compare with the monthly rainfall totals from the products. Fig. 5 displays bar graphs of 
the statistical results for each of the products using monthly totals for each of the months of 
1992. Of the twenty-five observational products, nineteen had a negative bias compared- to 
the atoll data. When limited to the Quasi-standard products, the data set had eight out of 
nine products having a negative bias, with the ratio being 0.88. This result is in contrast to 
that of a previous intercomparison (AIP-3) in the Western Pacific Ocean which used 
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surface-based radar data from the TOGA-COARE program as its validation (Ebert, 1998). 
There a very large majority of products (including many of the same retrieval schemes 
represented here) had a positive bias. Although these two validation data sets are in 
different locations and use different measurement methodologies, this difference needs to 
be addressed before we can be confident of the absolute magnitude of the rainfall estimates 
over the tropical oceans. 

As a group, the Q-S products have a higher correlation, a lower root mean square error 
(RMSE) and a smaller bias than the Experimental products. The models as a group have a 
reasonable bias, but very low correlations, indicating that they are less accurate than the 
observations in portraying the spatial and temporal variations of the monthly fields on these 
scales over this portion of the tropical ocean. Of the standard products the NOAA Merged 
Product ( nmg ) has the best statistics; however, it uses the atoll information in its merger 
process, so these statistics may not reflect what would be the results in other locations 
where there are no atolls. The venerable gpi has the highest correlation of any satellite- 
information-only product, reemphasizing the importance of sampling. Although the gpi 
also has a very small bias error against the atolls, the zonally-averaged, oceanic results 
again point to its limitations in sub-tropical regions (a large positive bias), as is well know 
to its users. Two of the Q-S products, the tov and nmi have relatively large biases. 

Of the Experimental products, the OLR Precipitation Index ( opi ) also exhibits very good 
statistics, although it has been derived by correlating against the nmg product where 
gauges are used. These validation statistics may therefore not be indicative of the accuracy 
of the product in locations without gauges. Of the SSM/I-based products (Experimental or 
Quasi-Standard) the bite and gem have the lowest RMSE, with a number of other 
retrievals close behind. The SSM/I-based products seem to have an upper limit to the 
correlation of about 0.75 because of the sampling limitation of the polar orbit, even though 
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almost all of the SSM/I products used data from two polar orbiting satellites, therefore 
maximizing the sampling. 

4.3 Validation of estimates over land 

Validation over land for the months of 1992 was carried out using the raingauge data sets 
previously described. Products that directly incorporated raingauge information {gpm and 
nmg) performed best statistically among all the products. Even the Experimental (EXP) 
product opi, which is derived through comparison with the nmg provides very good 
statistics. However, caution must be used in evaluation of these statistics. Because we are 
validating the products in locations where the raingauge information is of high quality and 
plentiful, these statistics may be overly optimistic as to how these products perform in 
general, especially in areas of poor raingauge coverage or quality. One example of the 
impact of the raingauge data can be seen by comparing the gps and gpm products. The gps 
is a merged satellite data product, whereas gpm additionally incorporates the gauges. In the 

tropical belt (30 ° N - 30° S) the rms drops from 56 to 30 mm when gauges are added, 
nearly a 50% reduction. 

The comparison of combined 12 month statistics also indicates that the climatologies are 
competitive with the satellite-based and model products. This result is related to the use of 
only one calendar year for this comparison and the large variance in the rainfall data set 
related to the climatological spatial and seasonal patterns. Only the products incorporating 
the raingauge data for the particular year had significantly better validation statistics than the 
best of the climatologies. 
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In the tropics over land (30°N - 30°S) the Q-S products and the climatologies as groups 

have the best RMSE, with the models generally having a positive bias and a larger RMSE 
(see Fig. 6). Of the observational products without influence from gauge information, the 
gps has the lowest RMSE. Over land the gps is a merger of the nmi and gpi and gives an 
RMSE of 58 mm as compared to the RMSE of 91 mm for the nmi and 77 mm for the gpi. 
In this region five observational products not affected by gauge information have ratios 
between 0.80-1.20 and correlations greater than 0.75. These are bup, cuf, cul, pur, pic, 
gps. Of the SSM/I products the PIP-1 composite (pic) has the lowest RMSE of 61 mm. 


In mid-latitudes (30°N - 60°N) the models and even the climatologies outperformed the 

observational products, except for those using raingauge information (see Fig. 7). The 
models had small biases and high correlations, with the ECMWF model (ecm) having the 
best RMSE (31 mm) of all products not using gauge information. In this region, when the 
same ratio and correlation criteria as in the previous paragraph are used, no non-gauge 
products meet the criteria. If the criteria are loosened, to 0.75-1 .25 for the ratio and greater 
than 0.65 for the correlation, then buc and bus emerge. 

When the latitude boundaries are expanded to encompass the entire 60°N-60°S region (not 

shown) the buc, bup, pur, gps and pic products meet the original criteria. It should be 
remembered that the SSM/I-based products do not typically have estimates over portions of 
the winter hemispheres over land due to cold land/snow contamination and that the statistics 
shown are for a matched set, using only those points where all the algorithms produced 
estimates. 

In a footnote related to sampling the relatively poor showing by the nmi product in the 
tropics over land (RMSE of 91 mm) is due mainly to the use of only one satellite in this Q- 
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S product. The other SSM/I-based products used two satellites in producing their 1992 
estimates. When nmi was run with two satellites after the workshop the RMSE for the 
tropical land area was reduced to 71 mm, which is comparable with most of the other 
SSM/I products. This very significant difference emphasizes the importance of sampling in 
the production of monthly precipitation estimates. 

In general, these statistical results for 1992 over land indicate that in the tropics a number of 
observational algorithms produce good results, but that over mid-latitude land results 
degrade, both absolutely and relative to the model calculations. This is especially true for 
the microwave-based products. The addition of gauge information greatly increases the 
accuracy of the products. 

5. Interannual results 


An evaluation of product performance related to the estimation of inter-annual variations 
was performed using the Januarys and Julys of 1991, 1992 and 1993. However, due to 
the near total absence of SSM/I data from January 1991, the difference fields between 
January 1991 and January 1992 were eliminated from the statistical comparison, leaving 
one January difference field and two July difference fields. The SSM/I products used for 
the interannual comparisons were produced using data from only one DMSP satellite. 
Example interannual difference maps between January 1992 and January 1993 are shown 
in Fig. 8. The strong El Nino in 1992 produced a strong mid-Pacific Ocean maxima in 
January 1992 and that is reflected in the interannual difference fields. All four example 
fields have very similar patterns over the tropical Pacific Ocean and to a lesser degree 
elsewhere in Fig. 8. Subtle differences are evident over the Indian Ocean and in mid- 
latitudes. 
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Statistical results over the Western Pacific Ocean region using the atoll data set indicate that 
as a group the Q-S products performed well with correlations in the 0.75-0.80 range and 
relatively low RMSE, with only the PIP-1 composite (pic) doing poorly (see Fig. 9). This 
low correlation for the pi c product may be due more to the one satellite sampling in the 
interannual exercise. The other Q-S products either have the better sampling related to the 
geosynchronous satellites or multiple polar-orbiters, such as with the tov and msu 
products. Among the EXP products the results are very variable with the opi and a few of 
the SSM/I products having reasonable correlations, approaching or equaling the validation 
statistics of the Q-S products. However, those SSM/I products that have relatively good 
interannual statistics with the atoll data are generally not among the best on the monthly 
statistics discussed earlier. Thus, taking into account both the annual and interannual 
statistics over the atolls, there is no clearly superior product among the SSM/I-based 
entrants. 

The models did poorer than the observational products as a group in terms of the 
correlation (0.4-0. 6), as expected, with the ecm (ECMWF) results having the highest 
model correlation. The ecm also has the highest rmse, indicating that this product is not 
reproducing the magnitude of the interannual differences. In the 1992 monthly statistics of 
the previous section, ecm was roughly comparable or a little worse than the other models. 
Thus, combining the information in both the monthly and interannual statistics leads again 
to the fact of no clear distinction among the model products. 

Over land areas in the Tropics (30°N-30°S) the products directly using gauges easily do 

best (see Fig. 10). Again it should be stressed that these statistics are for areas where the 
validation data, and therefore the input data to these products are the best. Again the Q-S 
products generally provide the best answers with better correlations and RMSE’s than the 
experimental products or models. Among the Q-S products that do not include gauge 
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information, the gpi has a high correlation, but an RMSE about the same as gps and tov . 
The PIP-1 composite product (pic ) does relatively poorly in terms of both correlation and 
RMSE. In fact all the SSM/I-based products have poorer statistical results than the Q-S 
products (other than pic). The opi product has interannual statistics in this region roughly 
equivalent to the satellite-only Q-S products, a drop-off compared to its better relative 
statistics on the monthly rain totals. The model products have lower correlations and 
higher RMSE’s than most of the observational products. 

In mid-latitudes (30°-60°N) over land the interannual statistics (Fig. 1 1 ) indicate the 

models approximately matching the quality of the observational estimates, with the 
exception of those that include the raingauge data. The median model correlation of 0.65 is 
equal to or better than all the values for the SSM/I-based products and approximately 
matches that of the non-gauge Q-S products and the opi. This interannual result is similar 
to that found with the monthly statistics in mid-latitudes. 

Figure 12 shows examples of interannual change during July in four example areas. 
Examples from January are not used because of the lack of SSM/I data in January 1991. In 
this exercise data from 4-6 adjacent or nearby boxes were combined to reduce the algorithm 
and validation random error. In Fig 12a the three year July variation over the six boxes of 

the atoll data set between 5°N and 10°S is shown for the pertinent products that produced 

results for two or three of the months. Most of the observational products reproduce the 
interannual changes in the atoll gauges, especially the relatively large change from July 
1.992 to July 1993, which is partially related to the end of the 1992 ENSO event. The three 
model calculations do not fare as well, although the ecm reproduces the tendencies 
correctly, but not the absolute values of the precipitation. 
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Two nearby areas over land in summer are shown in Figs. 12b and 12c. In Fig. 12b 
results for a 5°by 5° latitude-longitude area (37.5°-42.5°N, 87.5°-92.5°W) centered over 

the state of Missouri in the United States show a large increase in rain leading up to the 
heavy flooding in 1993 with mean July rainfall in the area of over 300mm. The 
observational products as a whole do not reproduce the interannual change correctly. The 
microwave-based products all overestimate the magnitude of the change, sometimes very 
substantially. This may be due to the microwave algorithms mis-identifying wet ground or 
standing water as falling precipitation. Infrared-based products, such as gpi and opi do 
better than the microwave products, but seem to saturate, and do not identify the increase 
from 1992 to 1993. The three models, on the other hand, reproduce the raingauge results 
in this location very well. A different pattern of interannual changes is shown in Fig. 1 2c 

for an area to the southeast covering part of the state of Mississippi (32.5°-37.5°N, 85°- 

90° W). With much smaller magnitudes (note the scale difference in the diagrams) and a 

sharp decrease between 1992 and 1993, the observational products overall do a very good 
job in this area, while two of the models indicate poor results. 


Over southeast Australia (32.5°-37.5°N, 145°-150°E) in the cool season (July), Fig. 1 2d 

indicates that most of the satellite products capture the interannual variation qualitatively, 
but a number underestimate the magnitude of the precipitation. The gpi matches the 
validation nearly perfectly, while the opi seems to underestimate the magnitude of the 
changes. The models also capture the time change, but underestimate the precipitation. 

The four examples shown are not all encompassing. However, the results indicate that 
both for the observational products and the models as groups, and for individual products, 
the interannual results are very variable in quality and therefore these products should be 
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used with caution when assessing interannual change. The results over land also indicate 
that including gauge information, where available, is critical. 

6. Frequency of oceanic precipitation 

Validation of monthly rainfall totals over oceans is limited to the atoll data set as 
representative of the open ocean. This limits the geographic scope of a validation exercise 
to a portion of the Western Pacific Ocean. In order to obtain measures of how well the 
precipitation products in this intercomparison were reproducing the observed precipitation 
distributions over the oceans generally; the product producers were asked to provide 
estimates of precipitation frequency in addition to monthly rainfall totals. These estimates 
could then be compared to the precipitation frequencies derived by Petty ( 1 995). Examples 
of maps of the annual frequency of precipitation from the submitted products and from the 
COADS data are shown in Fig. 13. The upper left panel in Fig. 13 shows the COADS 
climatology from Petty (1995). The main tropical and extratropical maxima and minima are 
evident with some boxes in the central Pacific ITCZ reaching 15-18%. Elsewhere in the 
tropics there are significant areas with rain frequencies of 8-10% in the climatology. In 
mid-latitudes of the Northern Hemisphere, the COADS data indicate maxima of 8-10%, 
increasing to 14% at 60°N in the Atlantic. The two experimental observational products 
shown reproduce the main features with significantly different magnitudes (from each 
other) in both the tropics and mid-latitudes. For example, in the central North Pacific one 
technique shows 2-4%, while the other product has 8-10% over a large area. This 
difference is typical of differences shown among the SSM/I-based products. The example 
model product in the lower, right panel of Fig. 13 has very large frequencies in the tropics 
and frequencies in mid-latitudes more comparable to the validation data and the other 
products shown. This difference in the model product between tropics and mid-latitudes is 
partially due to the convective nature of the precipitation in the tropics compared to the 
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widespread precipitation in mid-latitudes. This difference and the relatively coarse mesh of 
the model may help produce the initial result in Fig. 13. 

The frequency of precipitation comparison was very useful in delineating the 
reasonableness of the precipitation patterns and in some cases the accuracy of the products. 
However, the quantitative usefulness of the comparison is muted by the sensitivity of the 
results to the rainfall rate threshold used in defining the rainfall frequency both in the 
satellite and model products and the COADS surface observations. For this study the 
frequency of precipitation from the COADS data was defined as light precipitation and 
heavier (eliminating drizzle) at the station and at the time of observation. The product 

producers were advised to use 0.5 mm/h as a threshold, if they had that flexibility in their 
product. 

The elimination of drizzle from the COADS frequency of precipitation values may produce 
an underestimate of the actual precipitation frequency, when compared to the threshold of 
0.5 mm/h, especially in middle and high latitudes. This can be seen in Fig. 14 where a 
scatter plot of human observations of precipitation (excluding drizzle) are compared with 
raingauge measurements of 0.5 mm/h or greater over the United Kingdom. The results 
indicate a bias of approximately 3%, with the human observation (equivalent to COADS) 
higher than the raingauge values (presumably comparable to the precipitation products in 
the intercomparison). Use of a raingauge threshold of 0.2 mm/h eliminates most of the 
bias. Therefore, this sensitivity of the results to the rainrate threshold and the magnitude of 
the bias should be kept in mind when evaluating the following results. 

The results indicate a very large variability among all the products in the general oceanic 
precipitation frequency. Some of the large variation stems from various satellite and model 
footprint (or grid resolution) sizes and in different precipitation rate thresholds used. The 
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most homogeneous data set is that coming from the SSM/I-based algorithms, although 
among this subset there was still considerable variation (see Fig. 15). The zonally- 
averaged, annual (for 1992) peak in the ITCZ shows values ranging from 4% to 13% 
among the SSM/I products, with the COADS climatology indicating 8%. The SSM/I 
results are approximately equally distributed above and below the COADS value. In the 
Northern Hemisphere sub-tropical minimum the COADS climatology shows 3%, while the 

majority of SSM/I values are lower. Moving poleward from 30°N the COADS values 

increase continuously from 4% at 30N to 1 1 % at 60°N. Many of the SSM/I algorithms 

produce lower estimates of the precipitation frequency throughout this zone, although a few 
compare favorably with the COADS up to 45N, where they peak and then decrease rapidly, 
producing a very pronounced underestimation. The possible ~3% high bias in the COADS 
estimates would reduce the difference, but the latitudinal profile would still not be similar. 
This middle and high latitude underestimate is probably related to a failure to detect frozen 
precipitation and the relatively light rain at these latitudes as can be inferred by examining 
the second, lower COADS curve in high latitudes which excludes frozen precipitation. A 

few of the SSM/I products (r^, pur, gpf) do reasonably well up to 50°N in this 
comparison. 

Among the non-SSM/I satellite products there is a general overestimate of the precipitation 
frequency. Both the tov and msu products have an ITCZ zonally-averaged annual peak of 
about 30%, a Northern Hemisphere subtropical minimum of 8% and 45N value of 18%, all 
more than double the COADS values. On the other hand, both of these products show an 
underestimate in total rainfall in relation to the atoll data, indicating a relatively small 
rainrate in the raining areas of these products. These large values of rainfall frequency may 
partially be related to the larger footprint (relative to SSM/I) of tov and msu retrievals 
and/or the threshold used by the product producers. The gpi also has high values of 


21 



frequency, especially in the ITCZ, because of the use of an infrared T b threshold of 23 5K, 
which gives the best correlation with rainfall occurrence over large areas and long times, 
but produces a cold cloud” area usually much larger than the rain area. 


Two of the model-based products ( geo and ncp) submitted frequency of precipitation 
information. The zonal-averaged precipitation frequencies from these two models are given 
in Fig. 16, along with the COADS estimates. The model estimates represent the occurrence 
of precipitation somewhere in the relatively (to SSM/I observations) large grid box, so that 
the model estimates might be expected to be an overestimate compared to the COADS 
frequency. The model-based precipitation frequencies exhibit very distinct differences 
from the COADS numbers and the satellite estimates. The two models both have 
significant positive biases in the tropical rain areas and negative biases in mid-latitudes. 
The zonally-averaged, annual peak in the N.H. ITCZ for both the models is about twice the 

COADS value of 8%. In the S.H. there is a secondary peak at approximately 5°S with the 

geo frequency again being about twice the COADS value (5%); however the ncp value is 
significantly higher (15%), about three times the COADS value. This large overestimate in 
the ncp values, relative to both the COADS and the geo values is related to an eastward 
extension of the South Pacific Convergence Zone (SPCZ) into the central South Pacific 
Ocean and a western extension of the Indian Ocean maximum toward Africa that do not 
exist in either the COADS climatology or the satellite frequency estimates. The geo model 
product has a very weak eastern Pacific maximum, while the ncp nearly fails to produce the 
Atlantic ITCZ. 

In terms of geographic distribution of the frequency features most of the SSM/I-based 
products reproduce the locations of the maxima and their relative magnitudes (see Fig. 1 3). 
The western and eastern Pacific Ocean features are reproduced, as well as their seasonal 
movements. The advance and retreat of the Asian monsoon can be easily traced in the 
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rainfall frequency diagrams. In mid-latitudes the feature agreement is somewhat worse due 
to the apparent inability of the algorithms to capture the increase in precipitation frequency 
with latitude, although even somewhat subtle features, such as the maximum off the west 
coast of Canada and Alaska in winter is clearly defined. 


In terms of seasonal variation in the 30-45N region the COADS data show a distinct 
wintertime maximum that is only clearly reproduced in a few of the SSM/I-based products, 
namely pur and tam (see Fig. 18). Some other products fail to clearly retrieve this seasonal 
variation, or in a few cases even place the maximum in the summer. In the Southern 
Hemisphere mid-latitudes the annual frequency map from the COADS data show rainfall 
frequency maxima located southeast of Africa and South America and in the mid-Pacific 
Ocean where COADS data is sufficient for analysis. The satellite-based products that best 
reproduce both the location of the features and the approximate magnitude of the features 

are gPf pur and rss. The gme product also reproduces the location well, but greatly 
overestimates the magnitude. 


For seasonal variation both model products capture the annual variation in mid-latitudes 
with a peak precipitation frequency in the winter of the respective hemispheres, although 
both consistently underestimate the frequency. In the tropics the models show the general 
overestimate, as stated before, but the geo product has a small overestimate in the N.H. 
winter, but a large overestimate in the N.H. summer, primarily related to an apparent large 
overestimate of precipitation in the Western Pacific Ocean. The twelve-month correlation 
statistics, which indicate how well the products are delineating the spatial and seasonal 
variations over the ocean, show that in general the models have much lower correlations 
than the observational products in the tropics, but have correlations better than many, but 
not all of the satellite products in the 30-60N zone. Between the two models the 
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correlations show approximate equality in the mid-latitude zone, with the ncp having higher 
correlations in the tropics. 

The quantitative comparison of the SSM/I-based precipitation frequency products with the 
COADS data indicate some clear differentiation among the products. An examination of the 

annual average frequencies in the tropics (30°N-30°S) finds eight products (, cuf, cul, dir , 
gpf iow, rss, nmi, tam ) that have a bias ratio of between 0.70-1.30 and a correlation at or 
above 0.8. Expanding the zone to 45°N and 45°S reduces the list to gpf pur, nmi., rss, 

and tam . A table of results for the 45°N-45°S region is given in Fig. 17. Again 

expanding the area to 60N-60S, one is forced to loosen the requirements for the correlation 
to 0.70. Again the five products meeting these criteria are gpf, pur, nmi., rss, and tam, 
with the highest correlation (0.81) held by pur. One must remember that even as we 
expand the latitudinal zone the preponderance of the data set is in the tropics. Restricting 
the latitude band to 30-60N to examine the mid-latitude numbers, we find the highest 
correlation belongs to pur. 


Similar results, but with much poorer correlation coefficients, is evident if we use the 
monthly statistics instead of the annual means described in the last paragraph. However, as 

mentioned previously most products did not reproduce the seasonal cycle in the 30-60°N 

band and this shows up clearly in the correlation statistics, where pur had a significantly 
higher correlation than any other SSM/I-based product. 

Among the non-SSM/I products, the msu does well in terms of correlation coefficient in the 
tropics (30 N-30°S), equaling the best SSM/I value there. However, in the expanded 
latitude zones it loses significant ground to the SSM/I values. The tov correlations are 
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lower than the leading SSM/I-based values in all latitude zones. The gpi, surprisingly, also 
does poorly compared to almost all the SSM/I values in the 30°N-30°S zone, with a 
correlation of 0.60 for the monthly statistics. 


7. General Conclusions 

The intercomparison of the many observational and model-based precipitation products in 
this effort results in a plethora of images and statistics. However, because of the 
limitations of the validation data in terms of both coverage and quality and because of 
products often performing well in only certain locations or situations, there is great 
difficulty in unambiguously pointing to a certain product as “best” in terms of a monthly 
precipitation total over most of the globe. However, general conclusions can be drawn 
from the intercompanson results and recommendations to both the producer and user 
communities can be made. 

The intercomparison reveals a very large range of estimates among the products. Even the 
zonally-averaged, annual total field has a factor of two to three between the smallest and 
largest values, depending on the latitude. The range of values is reduced considerably 
when the observational products are limited to those from the Quasi-Standard set. This 
generally better performance by the Q-S products is also evident in both the atoll and land 
monthly validation and even the inter-annual results. The model-based precipitation fields 
do significantly poorer than the observational fields in the Tropics, but are competitive with 
the satellite-based fields in mid-latitudes over land. The inter-annual statistics do not 
necessarily follow the monthly statistics in terms of which products perform well, at least 
among the SSM/I-based products. The frequency of precipitation intercomparison was 
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very worthwhile in terras of gaining a better picture of how oceanic algorithms performed 

over a wide range of climatological zones. In terras of SSM/I-based products, the 

COADS-based frequency information clearly helps to diagnose the performance of the 
products. 


Some general conclusions from the intercomparison are as follows. This intercomparison 

clearly establishes the value of the merged analysis products that incorporate information 

from two or more satellite sources and blend in the raingauge data. Over land these 

products demonstrate superior statistics because of the incorporation of the gauge data. 

However, because the over-land evaluation centers on areas of plentiful gauge data, further 

evaluation of the products is needed using data exclusion tests to determine the error 

characteristics in areas of little or no gauge information. Even without the gauges there is 

evidence that the merger of microwave and geosynchronous IR data produce a better 

product. A second general conclusion is that the Quasi-Standard products generally 

outperform the Experimental group as a whole. This result is closely related to the maturity 

of the products. The Q-S products are, for the most part, products that have undergone 

substantial testing over a period of time, including in other intercomparisons. Many of the 

EXP products were being produced for the first time and will no doubt perform better in 
the future. 


Results from the Tropical Rain Measuring Mission (TRMM), launched in late 1997, should 
be a major source of new information on tropical rain totals, structure of tropical rain 

systems and how current rain estimation techniques compare with the improved 
information from TRMM. 


8. Recommendations to the User and Producer Communities 
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The PIP-3 Workshop and the related intercomparison activities resulted in an excellent 
review of the status of global precipitation analysis on a monthly time scale and, as always, 
raised a number of questions as to the large variability and reliability of estimates in certain 
regions and the validity of the validation or comparison in-situ data sets. The workshop and 
associated analysis did arrive at some general conclusions which are related to the 
following recommendations to the user and producer communities. 

Recommendations: 

1) For the period 1987-present, the user community should focus their use of global, 
monthly precipitation fields on the merged analysis products combining information from 
SSM/I data, geostationary IR data and raingauge data. Users should note possible 
limitations in these products, for example, a high latitude low bias over oceans (related to 
SSM/I estimates) and some observed artifacts in interannual fields. The over ocean values 
in these products are critically dependent on the SSM/I-based estimates, thereby 
emphasizing the need to use accurate, validated frequency of rain and rain amount 
information from the SSM/I algorithms. 

2) The apparent high latitude ocean low bias in most (not all) SSM/I precipitation total and 
frequency fields should be a focus of research attention with the objective of development 
of an approach which agrees more closely with the presumed correct precipitation 
frequency climatologies. Mechanisms for development of validation data sets in middle 
and high latitude oceans should also be pursued. 

3) Continued research and analysis should be done on the use of MSU, TOVS, OPI and 
other data sources for potential use in global precipitation analysis in the 1979-1987 period 
before SSM/I. 


27 



4) A product or products providing information on the variability among all (or a subset) 
of the observational products should be developed and analyzed to judge its utility as a 
measure of the unanimity of our estimates as a function of location and season. In the 
absence of high quality validation data this approach would give indications where 
estimates agreed and disagreed. 

5) The user community requires global precipitation fields at finer space and time scales for 
diagnostics, model validation (including diurnal variations) and assimilation into models. 
Requirements: ~6 hours, 100 km for global coverage. 

6) For future intercomparisons all products that include, gauges explicitly should have a 
non-gauge version in order to better intercompare satellite-only products. In addition 
frequency of precipitation should be included along with precipitation total with each 
submitted product so that the COADS frequency information may be used for direct 
comparison with all the products. 
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Table Caption 

Table 1. List of precipitation products along with short code for each product, source of 
data for each product, land/ocean coverage for each product and reference. 


Figure Captions 

Fig. 1. Location of 2.5° boxes of validation data. The shaded boxes were used in the 

validation of monthly amounts and the sources of data are described in the text. The 
outlined boxes were used for validation of interannual changes. 

Fig. 2. Four examples of monthly maps of estimates of precipitation for July 1992 
representing different types of products. 

Fig. 3. Zonal totals of precipitation for 1998. (a) the mean, maximum and minimum from 
the product data set as a function of latitude, (b) the standard deviation of the zonal mean 
among the observational products and among the Quasi-Standard products. 

Fig. 4. Zonal totals of precipitation for 1998. (a) the mean of the observed products 
compared to climatologies, (b) results from the four model products. 

Fig. 5. Statistical results for each of the products for monthly rain over the atoll region of 
the Western Pacific Ocean. The results are grouped by product type. 
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Fig. 6. Statistical results for each of the products for monthly rain over land in the Tropics 
from 30°N to 30°S . The results are grouped by product type. 

Fig. 7. Statistical results for each of the products for monthly rain over land in mid- 
latitudes from 30°N to 60°N . The results are grouped by product type. 

Fig. 8. Four examples of monthly maps of interannual change of precipitation from July 
1992 to July 1993 representing different types of products. 

Fig. 9. Statistical results for each of the products for interannual change of precipitation 
over the atoll region of the Western Pacific Ocean. The results are grouped by product 
type. 

Fig. 10. Statistical results for each of the products for interannual change of precipitation 
over land in the Tropics from 30°N to 30°S . The results are grouped by product type. 


Fig. 1 1 . Statistical results for each of the products for interannual change of precipitation 
over land in the Tropics from 30°N to 60°N . The results are grouped by product type. 

Fig. 12. Examples of interannual change in precipitation in four areas (one ocean, three 
land). Over land the areas are 5° by 5° latitude/longitude boxes shown by the outlined 

areas in Fig. 1. Over water the area is the sum of six 2.5° boxes in the outlined area in the 
Western Pacific in Fig. 1. 
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Fig. 13. Four examples of maps of estimates of precipitation frequency over ocean for 
1992 representing different types of products and the climatology based on COADS data. 

Fig. 14. Comparison of the frequency of human observation of precipitation (excluding 
drizzle) to the frequency of raingauge measured rainrates of 0.5 mm/h, or greater. 

Fig. 15. Comparison of the zonal average of precipitation frequency over ocean for the 
SSM/I products as compared to the COADS climatology. 

Fig. 16. Comparison of the zonal average of precipitation frequency over ocean for 1992 
for two model products as compared to the COADS climatology. 

Fig. 17. Statistical results for each of the products for frequency of precipitation over ocean 
for 1992 from 45°S to 45°N . The results are grouped by product type. 


Fig. 18. Seasonal variation of precipitation frequency over the ocean between 30°N to 45° 

for 1992 as estimated by various products and the COADS data and compared to the 
COADS climatology. 
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